Model - Averaged ` 1 Regularization using Markov Chain Monte Carlo Model Composition
نویسندگان
چکیده
This paper studies combining `1 regularization and Markov chain Monte Carlo model composition techniques for Bayesian model averaging (BMA). The main idea is to resolve the model uncertainty issues arising from path point selection by treating the `1 regularization path as a model space for BMA. The method is developed for linear and logistic regression, and applied to sample classification in four different biomedical datasets.
منابع مشابه
Feature Selection by Markov Chain Monte Carlo Sampling - A Bayesian Approach
We redefine the problem of feature selection as one of model selection and propose to use a Markov Chain Monte Carlo method to sample models. The applicability of our method is related to Bayesian network classifiers. Simulation experiments indicate that our novel proposal distribution results in an ignorant proposal prior. Finally, it is shown how the sampling can be controlled by a regulariza...
متن کاملMonte Carlo Hidden Markov Models
We present a learning algorithm for hidden Markov models with continuous state and observation spaces. All necessary probability density functions are approximated using samples, along with density trees generated from such samples. A Monte Carlo version of Baum-Welch (EM) is employed to learn models from data, just as in regular HMM learning. Regularization during learning is obtained using an...
متن کاملMonte Carlo Hidden Markov Models: Learning Non-Parametric Models of Partially Observable Stochastic Processes
We present a learning algorithm for non-parametric hidden Markov models with continuous state and observation spaces. All necessary probability densities are approximated using samples, along with density trees generated from such samples. A Monte Carlo version of Baum-Welch (EM) is employed to learn models from data. Regularization during learning is achieved using an exponential shrinking tec...
متن کاملImage Deblurring, Gaussian Markov Random Fields, and Neumann Boundary Conditions
In this paper we consider the inverse problem of image deblurring with Neumann boundary conditions. Regularization is incorporated by using Gaussian Markov random fields (GMRFs) to model an appropriate prior on the image pixel values. We provide a linear algebraic framework for GMRFs, and we establish an important connection between GMRFs studied in the statistical literature, and negative-Lapl...
متن کاملA Moreau-yosida Approximation Scheme for High-dimensional Posterior and Quasi-posterior Distributions
Exact-sparsity inducing prior distributions in high-dimensional Bayesian analysis typically lead to posterior distributions that are very challenging to handle by standard Markov Chain Monte Carlo methods. We propose a methodology to derive a smooth approximation of such posterior distributions. The approximation is obtained from the forward-backward approximation of the Moreau-Yosida regulariz...
متن کامل